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Part 1 
Foreword 


The first 5 papersin this RECORD on the guidelines for research design 
have been developed from a conference session at the 49th Annual 
Meeting chaired by George W. McAlpin. The general objective of the 
session and the papers was to bring out the analytical aspects of re- 
search design, with particular emphasis on experimental design and the 
application of statistical methods to controlled experiments and surveys. 
Irick presents the framework of the conference and identifies and dis- 
cusses 5 research design steps: systems analysis, information anal- 
ysis, program formulation, project scheduling, and experiment design. 
Anderson addresses himself to the design of experiments, outlining the 
fundamentals of statistically designed experiments and illustrating their 
application to a specific example. Hudson describes the applications of 
the principles to laboratory research projects, giving some examples 
and creating an interesting myth. Beaton identifies the basic categories 
of field experiments and discusses the application of design principles 
in this area. In the fifth paper, Finkner explores the use of statistics 
in planning sample surveys. 
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Research Design 
PAUL IRICK, Highway Research Board 


eIN MY BRIEF REMARKS I shall give a general background for the main subject, ex- 
perimental design. In essence, I shall try to present a framework for the major aspects 
of research design of which experimental design is a very important part. 


CONCEPT OF RESEARCH DESIGN 


I will start with the notion that research is the study of systems for the purpose of 
producing information about that system. Of particular corcern is the study of trans- 
portation systems, transportation subsystems, or systems closely related to the trans- 
portation field. I think that the main product of research is information that is useful 
for the operation, management, or further study of such systems. 

To increase the likelihood that useful information will, in fact, result from research 
activities, we believe that research must be designed; thatis, a rather complete set of 
plans needs to be developed for the research. To this end, we have identified 5 re~ 
search design steps (Fig. 1): systems analysis, information analysis, program formu- 
lation, project scheduling, and experimental design. 

I shall briefly discuss the first 4 of these steps. The other papers will be largely 
devoted to the fifth step. Because these research design steps are interrelated and 
highly interactive, it is important that those having responsibility for any one step be 
closely involved with all the other steps. 


SYSTEMS ANALYSIS 


We believe that research design should begin with a suitable analysis of the system 
that is to be studied (Fig. 2). A system is comprised of activities, inputs, outputs, 
utilities for conducting the activities, environments within which the activities occur, 
and characteristics of activities, inputs, outputs, utilities, environments, the system 
at large, and the related systems. A system analysis will generally include both the 
identification of all these system aspects and their interrelatedness. 

We would also ask that the system analysis name variables and attributes that may 
be used to describe the system. We may suppose that the system's independent vari- 
ables are those that describe system activities and elements and that system charac- 
teristics may be identified with dependent variables. 

A minimal result of systems analysis is an explicit or implied list of the types of 
information that are necessary and sufficient for understanding and improving the 
system under study. 


INFORMATION ANALYSIS 


Although the systems analysis should indicate information needs, it does not neces- 
sarily bring out the extent to which the needs have already been met. 

We believe the second step in research design should be an information analysis 
that reveals the existence and adequacy of available information on the system under 
study (Fig. 3). The acquired information can then be evaluated and analyzed with re- 
spect to information needs. The analysis should reveal information gaps and inade- 
quacies, particularly from the statistical point of view. We suppose that the net re- 
sult is a clear picture, perhaps in terms of problem statements, of information needs 
that might be satisfied through research activities. Thus we consider research needs 
to be equivalent to needed information on the system under study. 
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In passing, we observe that information analysis is frequently bypassed until the 
completion of program formulation and project scheduling steps—or even until the com 
pletion of the experimental designs. Thus we may find that literature searches may 
not be made until project investigators begin their work. Unfortunately, this practice 
sometimes brings out too late that the research design can only produce unnecessary 
or inadequate information. 


PROGRAM FORMULATION 


The third step in research design is to formulate one or more research programs 
whose goals cover at least part of the information needs. In this discussion we will 
suppose that program formulation is the structuring of research activities into pro- 
grams and projects within programs, and that program formulation includes the state- 
ment of program goals and project objectives (Fig. 4). Finally, we consider that 


SYSTEM ANALYSIS ERT & 
INFORMATION ANALYSIS 

PROGRAM FORMULATION ; 
PROJECT SCHEDULING TT PRREDATE 
EXPERIMENTAL DESIGN \ 


et 


RESEARCH 
DESIGN 


ae ar ACTIVITIES 
TRANSPORTA eimai 
INPUTS VARIABLES 
RESEARCH SYSTEM USEFUL pad ae ARIABLES 
SUBSYSTEM INFORMATION PUTED: & 
RELATED SYSTEM UTILITIES _ —_ 
ENVIRONMENTS ATTRIBUTES 


CHARACTERISTICS 


MANAGEMENT 
& OPERATION 
—— 


Figure 2. System analysis. 


Figure 1. Research concepts. 


NEEDS 


EXISTING 
INFORMATION 


PROGRAMS & GOALS 


——— 1 _____. 
: ( Bh ongel PROTECTS & OBJECTIVES 
SYSTEM SEQUISEEION ADEQUATE petit 
- EFFORT & PRIORITIES 


DESCRIPTION INFORMATION 


EVALUATION 


RESOURCES 


RESEARCH ‘ : 
Figure 4. Program formulation. 


NEEDS 


Figure 3. Information analysis. 


RESEARCH 
DESIGN 


RESEARCH 
NEEDS 
PROJECT 
SCHEDULING 
| | 
EXPERIMENTAL LAB FIELD SAMPLE TESTS & 
DESIGN EXPERIMENTS| | EXPERIMENTS 


SURVEYS EVALUATIONS 
Figure 5. Project scheduling. Figure 6. Research and experimental design. 


SYSTEM 
ANALYSIS 


INFORMATION 
ANALYSIS 


PROGRAM PROJECT 
FORMULATION SCHEDULING 


PROJECT 
TASKS 

EQUIPMENT 
PERSONNEL 
TIMING 


EXPERIMENTAL 
DESIGN 


DATA 
_ ACQUISITION 


3 


program formulation should include levels of effort and priorities for program and 
projects. 


PROJECT SCHEDULING 


Our fourth step for research design is project scheduling that identifies project 
tasks and personnel, equipment, and timing considerations for the conduct of project 
tasks (Fig. 5). Project scheduling thus provides management information for research 
administrators and constraints for the experimental designs that will be used to acquire 
the information. 


SUMMARY 


In summary, we have supposed that research design involves 4 steps that need to 
be accomplished in one degree or another before definitive experimental designs are 
developed for the acquisition of research information (Fig. 6): system analysis, in- 
formation analysis, program formulation, and project scheduling. We hasten to re- 
peat, however, that these steps are not independent of one another and their develop- 
ment should proceed in full consonance with the development of experimental designs. 


Design of Experiments 
VIRGIL L. ANDERSON, Purdue University 


® RESEARCH DESIGN may be considered closely allied to the scientific method. Kemp- 
thorne (1) describes the scientific method in a circular notion as follows: 


Prediction of Abstractions of essential 
new events elements to give basis for 


logical theory 


Development 
of theory 


The uses of statistics comes in the observations by designed experiments and anal- 
ysis of data from the experiments. The design should never be considered before the 
analysis is contemplated. 


FUNDAMENTALS OF STATISTICALLY DESIGNED EXPERIMENTS 


Ingredients for Well-Designed Experiments 


The first of these ingredients is to have a well-defined problem. This may take 
many hours, but in general an hour spent here usually saves at least ten later. In gen- 
eral researchers should not run one-factor-at-a-time experiments, and complete 
factorials are too expensive. Hence fractional factorials and modifications must be 
considered. 

Another ingredient is to have enough replications to show "practical" significance. 
The notion of practical significance must be associated with cost. There is no reason 
to run an experiment to show statistical significance (in a probability sense) when the 
difference of the means is so small that it does not pay the engineer to make a change 
in practices. This concept can be incorporated in the experiment at the design stage 
if discussed thoroughly. 

Two additional ingredients are to outline the analysis of the data at the design stage 
and to design the experiment as economically as possible. 


Basic Requirements for Designs 


In the past, the 2 requirements given were randomization to allow for unbiased esti- 
mates of the parameters of interest and replication to allow for an estimate of the 
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experimental error. Somehow, the engineers who have worked with me recently seem 
to know these, but 2 requirements are not understood. 

Inference Space—The concept is the same as that of the statistician's old word 
"population", but engineers appreciate inference space better because it implies how 
far they can apply the results or what inferences they can make from the data taken 
from the designed experiment. 

Restrictions on Randomization—In general, research workers in all fields in which 
I have consulted have difficulty understanding that, when a restriction on randomization 
is placed on the data-taking mechanism, this peculiarity must be accounted for in the 
analysis. It has been my experience over the past 10 years or more that almost no 
truly factorial experiments are run; they almost all have a tinge of "split plot'' in them. 
I do not expect you to understand this completely here, but I will try to demonstrate it 
later. When canned factorial computer programs are run on your data, you will more 
likely misunderstand the results for your experiment than understand them if you do 
not understand restrictions on randomization. 


Methods of Handling Extraneous Variables 


Rigid Control—The inference space may be too narrow for you, but for some experi- 
ments this is desired. 

Classification—This includes the treatments or factors, the blocks or restrictions 
on randomization, and anything else that has to do with controlling the levels of the vari- 
ables to be included in the experiment. 

Concomitant Information—Measuring another variable at the same time as the one of 
interest allows additional information with few observations. 

Randomization—After rigid control, classification, and concomitant information have 
been used to control or measure extraneous variables to obtain unbiased estimates, 
randomization is used to scatter the influences of those extraneous variables missed by 
the first three. 


Prior Information 


Some statisticians do not like to use information from prior investigations, especially 
if randomization has been neglected. There are occasions in engineering problems, 
however, when prior information is necessary before an experiment can be run intelli- 
gently. If care is taken, I believe historical data can be utilized. 


AN EXAMPLE 


Consider the problem of deciding on the best prosthetic cardiac valve to choose from 
4 types at 6 different pulse rates. An apparatus constructed by an engineer used water 
in the mechanical device for blood, a pump for the heart, and a certain type of tubing 
for blood vessels. The valves could be installed and replaced in the system with some 
difficulty. The pulse rates could be simulated by varying the speed of the pump quite 
easily. 


Complete Factorial 


If all 24 combinations of the valve types by pulse rates could be repeated 2 times, 
there would be 24 times 3 or 72 treatment combinations to run. If all of these 72 were 
run completely at random, as is assumed in a complete factorial, the engineer would 
have to disassemble his machine 71 times. The linear model to use for the analysis 
could be 


Y=u+V+P+VP+.e 


where 
Y = the variable to analyze such as back pressure or a flow variable, 
uw = the overall mean, 
V = the effect of the valve type, 


P 
vP 
€ 


the effect of the pulse rate, 
the interaction, and 
the error for testing the effects of V, P, and VP. 


tou ul 


The amount of information available to investigate the valve types is associated with «, 
which has 48 degrees of freedom. In this case the canned factorial computer program 
is appropriate. 


Split Plot Design 


Almost no engineer would take the time to disassemble the apparatus 71 times for 
such an experiment. He may be willing to put a valve type in the apparatus and run all 
6 pulse rates at random. Then he may remove that valve type and put another one in 
at random. Then he may remove that valve type and put another one in at random and 
run all 6 pulse rates at random. He may do this for the other 2 valve types and repeat 
this procedure with different randomizations for 2 more times. In this case he has re- 
stricted his randomization by allowing the same valve type to receive all pulse rates 
before disassembling the machine. Somehow this must be accounted for in the analysis. 
One linear model that could show this, if certain assumptions are made, is 


Y=yp+V+6+P+VP+e’ 


In this case 6 has been added in the model to account for the randomization restriction 
on valve types, and V is tested by 6, which has only 8 degrees of freedom. Also ¢’ has 
only 40 degrees of freedom, which is not much loss. 

The important point to engineers is that, by making the experiment easier to run, 
the information on valve types, per se, may be reduced and all this must be accounted 
for in the analysis. In this case the canned factorial computer program is wrong and 
must be modified for a correct analysis. 


SUMMARY 


In the example, rigid control was used as much as possible by the engineer, classi- 
fications were the valve types and pulse rates, concomitant variables could be the back 
pressure or a flow variable, and randomization was used. 

Most important, however, was that the inference space was the human beings to use 
these prosthetic heart valves. How immediately applicable the results would be must 
be ascertained by the engineers, medical doctors, and statisticians. Also, the restric- 
tions on randomization must be accounted for if the second design (a split plot) were 
used. Care must be given to the analysis of such data. 
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Statistical Experiment Design of 
Laboratory Experiments 
W.R. HUDSON, University of Texas 


°IN BOTH the university and in the real world one often sees engineers and students, 
and professors for that matter, working with test data, usually plotting them in an effort 
to find out what they think the data are telling them. In doing this, we often massage 
the data in various ways. We may find it desirable to throw out "bad" data points or 

to bias the curve so that it looks good to us. In doing this we ought to keep in mind 

2 things: 


1. We can do anything we desire to the data as plotted on that graph and we can do 
anything we desire with the shape of the curve, and our judgment may make the line 
better or worse. 

2. We are not usually interested in those data or that line because we will never 
use those data again. They are samples from the real world. What we are really in- 
terested in, usually, is what we can expect to happen under similar conditions either 
in the laboratories or in the outside world. In this case, what we do with the data or 
with the line makes quite a bit of difference because what the "truth" is or what nature 
intends to do will not be affected one bit by what we say about the data or the line. 


I think this is best illustrated by the following example. I walked into a research 
laboratory one day and observed a friend of mine carefully plotting 3 points on a graph 
like that shown in Figure 1. The graph represents the results from a complex test 
(Test A) compared to the results of a very simple test (Test B) for many pairs of tests. 
Observe the perfect correlation line that he already had plotted and the large number 
of points that seem to fit the line very closely. I observed my friend plotting from his 
data sheets the 3 points shown in Figure 2 by crosses through dots. As Ientered the room 
he seemed to be very disturbed. I said, 'Friend, what is the matter ?" He said, ''Oh 
heck, I thought I had 3 new points for my test correlation, but as you can see I have 
only one.'' With that he erased the 2 external points. There is no question that the set 
of points does represent a group of soils for which the 2 tests did correlate well. 
Figure 3, however, shows the set of all correlations run by my friend. He has thrown 
away all the points not shown on Figures 1 and 2. Unfortunately, he proposes to use 
the correlation in all circumstances, that is, to substitute Test B for Test A as his 
specifications test. I think that you can easily see the error in this approach. He 
threw away the unwanted points, but nature will not throw them away. 

There is another area of major concern in our laboratory work at a different level. 
Many of us do not treat our data as shown in Figure 1. We do, however, divide very 
complex problems into parts and look at them with a one-factor-at-a-time approach, 
ignoring the effects of the other variables and the resulting interaction. I would like to 
illustrate this with a little myth. Now remember that a myth while not literally true 
represents a truth. 


Once upon a time there were 3 boys. Their names were Ronnie Hudson, Johnny Beaton, 
and Isaac Newton. Now, Ronnie lived down in Texas where lemons and grapefruit are grown. 
As he rested under the trees, he noted with his high intelligence that grapefruit falling on his 
head from the tree hurt much more than lemons. Being very astute and very scientifically 
minded, he ran several experiments involving objects and the resulting force with which they 
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struck his head. That year he went to the Fruit Research Board and reported that empirically 
grapefruit were more dangerous than lemons. After several more years of study, while obtain- 
ing a very sore noggin, he also refined his data as shown in Figure 4. From this he deducted 
Hudson’s Law: Force is equal to a constant of 3 times the mass of the falling object. He 
dutifully went back to the national convention and reported these findings. 

Not long thereafter young Johnny Beaton, living out in California, lay around under the 
date trees and grape arbors. He noticed with brilliance equal to Hudson’s that when dates 
falling from tall trees struck his head they hurt much more than grapes falling from the low 
arbors. He also ran a scientific experiment and reported that the height of the tree was very 
important. After studying the data (Fig. 5), he noted carefully that the force of a falling 
object is really equal to a constant, 5 times the acceleration of the object. He dutifully went 
to the national convention and reported Beaton’s Law, which seemed to discredit, by in- 
nuendo although not directly, Hudson’s Law. 

Shortly thereafter young Newton, who lived somewhere in the northwest among the apple 
trees, often sat to think under small apple trees and large apple trees, under trees growing big 
apples and little apples. He noted very astutely that the pain due to an apple hitting his head 
depended on the size of the apple as well as the height of the tree; however, the problem was 
too complex for easy solution. He promptly called in 3 of his friends (Paul Irick, Virgil Ander- 
son, and Jack Youden) who considered the problem with him in some detail. Finally, an ap- 
propriate experiment was designed and apples of various size were dropped from various 
heights onto poor Isaac. Figure 6 shows very succinctly the data that resulted. Lo and be- 
hold, the mass really did not govern the results directly nor did the acceleration. Newton’s 
Law was therefore born: Force is proportional to mass times acceleration. When this re- 
sult was reported at the national convention, Beaton and Hudson were discredited, and it 
was reported that they subsequently have spent their declining years playing with dirt, ce- 
ment, and asphalt. 


To me this myth illustrates why our literature is full of contradictory conclusions. 
Like the blind men and the elephant, each of us deals with one part of the problem, such 
as the mass, while ignoring an equally important part, such as the acceleration. 

A serious example is shown in Figure 7. If I looked only at an asphalt content of 7 
percent, I would report that fine gradations are better than coarse gradations. How- 
ever, at an asphalt content equal to 4 percent, the exact reverse is true. At a recent 
conference, a colleague of mine reported that the tensile strength of asphaltic mixes 
did not depend on gradation. Figure 8 shows the experiment design that led to his 
conclusions. In the meantime, we ran a rather large experiment involving 7 variables, 
one of which was gradation. The results of these tests show that, in truth, gradation 
has no direct effect. Figure 7 shows, however, an important interaction effect, and 
the tensile properties do vary with gradation because the effect of asphalt content changes 
as gradations get finer because more asphalt is absorbed by the mix. 

Table 1 gives the factor space over which an asphalt stabilization experiment is to 
be conducted. Seven factors are includedat 2 levels such. Figure 9 shows another pre- 
sentation of the same factor space. Inthe figure, a better relationship can be seen among 
the variables because all possible combinations of the variables are shown and there 
is one block for each combination inthe factorial. Furthermore, this form can be used 
to collect data. The answers for a specimen applicable to a particular set of factors 
can be written down in that block. If the rows and columns are summed, you have a 
quick look at the data, and it is possible to make quick estimates of factor effect. 

If it is impossible to conduct as many tests as you desire, such as in the case of 
the factorials shown in Figure 9—128 specimens plus some replicates—we can run a 
special experiment taking some carefully selected samples. This is called a fractional 
factorial. Figure 10 shows a one-quarter fractional factorial, that is, one-quarter of 
the total number of blocks. It is not always desirable to take partial factorials; how- 
ever, under certain circumstances they can be especially useful in cutting down on the 
size of an experiment while obtaining satisfactory results. 

If a quarter fraction is too small, then a half fraction, as shown in Figure 11, can 
be tried; this is the first quarter plus the addition of a second quarter. You can see 
in Figure 11 the symmetry in the selection of samples to be taken. There are equal 
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numbers of all factors involved in the samples to be chosen so that there is balance 
within the experiment. Certainly there is not as much information as in a full factorial, 
because there are many fewer degrees of freedom in the experiment. 

Table 2 gives the factors in a cement treatment experiment involving 5 factors at 
3 levels and 4 factors at 2 levels each, or 5,832 specimens, an almost impossible 
number to deal with. In this case it is almost essential to deal with a one-quarter or 
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even a one-eighth fractional factorial. As a matter of fact, we ran an even more com- 
plicated experiment by breaking the experiment up into 3 separate parts, but that is 
too complex to discuss here. 

Let us turn our attention now to the results of some of these experiments. Figure 
12 shows what statisticians call a main effect. This is the effect that a particular vari- 
able, in this case molding water content, has on the results of a test, in this case, in- 
direct tensile strength. As we can see, increasing molding water content from 3 to 7 
percent increases tensile strength from 60 to 200 psi, We might write an equation for 
the effect in the form 


T=, <+Cw 


This is the type of effect that engineers most often model. 

Figure 13, however, shows a 2-factor interaction. Unfortunately, if we look at the 
main effects and think that we know what the effect of molding water content is on ten- 
sile strength, we will be badly mistaken, because molding water content also interacts 
with gradation to affect the results. Thatis, the effect of water content is greater on 
fine gradations than it is on coarse gradations. An interaction is the term we use to 
describe the cross-product terms in a mathmatical model when the effect of one factor 
depends on the level of another factor in the experiment. 

As if that were not complicated enough, there are sometimes 3-factor interactions 
present, which really causes problems. The 3-factor interaction involving water con- 
tent, cement content, and type of curing is shown in Figure 14. As you can see, the 
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TABLE 1 TABLE 2 
FACTORS INCLUDED IN ASPHALT FACTORS INCLUDED IN A CEMENT 
STABILIZATION EXPERIMENT TREATMENT EXPERIMENT 
Level Level 
Factor SS Factor 
Low High Low Medium High 
Aggregate type Crushed Rounded Molding water 
limestone gravel content, percent 3 5 7 
Aggregate gradation Fine Coarse Curing time, days 7 14 21 
Asphalt viscosity AC-5 AC-20 Aggregate 
Asphalt content, percent 5.5 8.5 gradation Fine Medium Coarse 
Mixing temperature, deg F 250 350 Type of curing Air dried = Sealed 
Compaction temperature, deg F 200 300 Aggregate type Gravel - Limestone 
Curing temperature, deg F 40 110 Curing 
temperature, deg F 40 15 110 
Compactive effort Low = High 
Type of compaction Impact - Gyratory shear 
. Cement content, 
effect of cement content is much stronger percent 4 6 8 


at high water contents than at low water 
contents, but it isalsostrongerin general 
for sealed than for air-dried specimens 
as shown by the greater height of the black columns. Thus we see that we cannot show 
the effect of water content alone, because it causes other effects in combination with 
other factors; in this case, cement content and type of curing. 
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I would summarize this discussion as follows: 


1. The design of experiments forces an explicit look at the problem, the factors 
involved, and the levels of interest. This look improves our understanding of the prob- 
lem. Ihave seen this so often with graduate students. They struggle for months to 
define and design their experiments, until they get a little exasperated with me because 
I do not doitfor them. After they get the experiments designed, however, they find 
that the completion of the tests and the final analyses become much easier. 

2. On complex problems, as most engineering research problems these days are, 
statistics is essential for getting the correct information from the test, getting the most 
information for your money in an experiment, and understanding quantitatively the find- 
ings of an experiment, that is, how good the answers are. 

3. On very complex problems, the assistance of a statistician can be helpful; in 
fact, it is sometimes almost essential. However, for many simple problems, andafter 
some consultations with a statistician, most engineers can, with the use of a good sta- 
tistics book and some hard work, because reasonably good at designing and analyzing 
simple experiments. 

4. Experimental design can be very valuable in conducting laboratory research 
experiments. 


Practical Problems in the Design of 
Field Experiments 


JOHN L. BEATON, California Division of Highways 


eCONSIDERATION of the basic principles of experimental design as well as the selec- 
tion of statistical methods for data acquisition and analysis is mandatory in the plan- 
ning phase of any experiment. This is true whether it is to be performed in the labora- 
tory or in the field. 

There are certain differences, however, between laboratory and field research that 
must be considered in the formulation of a field research program. These differences 
primarily result because the environment is beyond control of the researcher. How- 
ever, of almost equal concern are the management problems, particularly personnel, 
that occur in the field. If these factors are not recognized and considered in the ex- 
perimental design as possible sources of variation, their effects will most cer- 
tainly be confounded with the response of the material, condition, or property under 
investigation. 

This confounding of the effects of uncontrolled variables with controllable and mea-~ 
sured parameters under test will produce invalid quantitative data; and in addition, 
because of possible interaction among these effects, statistical tests may erroneously 
indicate that significant differences are occurring for the conditions being investigated. 
In reality, however, these differences may be insignificant. Usually, the effect of this 
sort of confusion is to limit the ability to detect changes in test parameters when they 
do occur. 


TWO BASIC CATEGORIES OF FIELD EXPERIMENTS 


At this point, a look at 2 basic categories of highway field research is in order. The 
first category includes those experiments in which the uncontrolled variables are 
largely confined to the intrinsic properties of the natural environment. Typical ex- 
periments might range from weathering of paints or signing materials to in situ test- 
ing of rock or soil for evaluation of new test devices. This category of experiment is 
usually characterized by a relatively relaxed time schedule. The investigator 
usually has direct control over project labor, administration, and progress. Many 
such projects are relatively inexpensive, and most can be repeated with little delete- 
rious change in the uncontrolled variables or increase in cost. 

The second basic category encompasses those projects that are associated with the 
construction of a highway facility or appurtenance. Typical work in this category might 
range from the instrumentation of an orthotropic bridge for observation of stress pat- 
terns to the measurement of pore pressure variation in a soft foundation due to embank- 
ment construction. Experiments in this category are often characterized by dependence 
on a contractor's time schedule. The investigator usually lacks direct control over 
project labor, administration, and progress. Such projects may be quite expensive 
and are usually impossible to repeat because the uncontrolled variables are unique to 
the particular location or structure and redoing an entire structure or even a portion 
would be prohibitively expensive. 

It is evident that research falling in this second basic category will require agreater 
safety factor, solely because of the irretrievable nature of the experiment. Bya 
greater safety factor I mean, for example, 1 or 2 levels of redundancy for critical 
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data points to guard against data loss due to failure, allow better statistical averaging, 
and provide better application of tests for significance. 

Some of the other problems occurring in this type of research are primarily ad- 
ministrative in nature. These administrative problems, however, do affect the design 
of the experiment. The effects are mainly financial and psychological. The psychologi- 
cal effects will be mentioned later under human factors as uncontrolled variables. 
From a financial standpoint, the administrative nature of such a project causes an ex- 
cessive expenditure of research money. This money is needed not only for extra in- 
strumentation but also for the extra man-hours for standby time to match the schedule 
imposed by the contractor or weather. 

Provisions for adequate working time to install the research facilities, physical 
help from the contractor's personnel, and subsequent protection of the research facility 
from the contractor's operations are a necessary part of the contract. The lead time 
for this type of research is, therefore, substantial and must be considered in the early 
planning stages. 

Once the contract is let, prosecution of the research is greatly facilitated by con- 
ferring jointly with key personnel of the contractor and the project engineer. Admin- 
istrative problems such as work sequence, payment for extra work, and joint and in- 
dividual responsibilities can be worked out. Some of the actual physical steps in the 
research should be kept flexible up to this point to allow incorporation of ideas that 
may be presented by the other conferees. If the researchers earnestly seek to enlist - 
the cooperation and interest of these people, the research will be greatly aided. Re- 
search personnel who like to project an "ivory tower" image will be a detriment to 
this sort of conference. 

An often justifiable expense for field experiments in this second category is the 
employment of a field coordinator for the research. This individual should be selected 
from the highway project personnel. He acts as a go-between for the researchers and 
personnel of both contractor and project once the project is under way. Mandatory 
personal attributes include statesmanship, an appreciation for research, a well- 
developed sense of humor, and an excellent knowledge of construction practice. Such 
a person can be worth many times his salary in smoothing the path for research 
personnel. 

During the progress of the research, both the project engineer and the contractor 
should be kept informed of significant and interesting developments. When the report 
is written, acknowledgment of their assistance should be made and they should receive 
copies of the report. Recognition of this sort will go a long way toward improving both 
present and future research climate. 


THE NATURE OF UNCONTROLLED VARIABLES AND OTHER PROBLEMS 


Uncontrolled variables affecting either or both of the 2 basic types of field experi- 
ments will normally fall into 1 of 4 general groups including environmental factors, 
human factors, hardware factors, and construction methods. Obviously, all of these 
variables will not apply to all field experiments and, in many cases, there will be 
considerable interrelationship among variables. 

Some of the problems, such as accidental or malicious damage to the experiment, 
cannot be dealt with statistically as can the unknown or uncontrolled variables. These 
problems must be considered, however, in planning the experiment. 


Environmental Factors 


Environmental factors are normally uncontrolled or only partially controlled in 
most field research. Included in this general group are obvious things such as tem- 
perature, relative humidity, wind, vibration, anisotropy of soil deposits, and corro- 
sive atmosphere or water. Also in this group are less obvious but diverse factors 
such as damage to the experiment due to falling rock or earth slope failures, biological 
damage to instrumentation, changes in water table due to pumping or irrigation, and 
changes in geometry with increase in fill height. These are just a few examples. 
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Human Factors 


In field research the investigator may have to consider the ability or desire of a 
craftsman to build a facility to the tolerance required for the experiment. Conversely, 
tolerances tighter than normal may be obtained because of the presence of a research 
observer. When the progress of the work is controlled by a contractor's operation, 
research personnel are often placed under stress because of unpredictable variations 
in the time schedule. Also, research that is part of such a highway contract is often 
a "poor relation" in the eyes of project and contractor's personnel. Morale and ef- 
ficiency of the research team suffer as a result. These and other similar psychologi- 
cal factors have a direct bearing on the reliability of the acquired data. 

Adverse working conditions such as rain, snow, and mud will doubtless affect 
the efficiency and performance of the researchers. Often the nature of the field ex- 
periment requires considerable assistance from personnel who are not research- 
oriented. In these cases, communication, or rather the lack of it, can lead to invalid 
data. Careful selection of research personnel and assistants is indicated. In addition, 
enthusiasm for the project must be instilled in everyone involved. 

Vandalism can be a nightmare to the researcher in the field. An average marks- 
man with a small caliber rifle can reduce an unprotected installation to junk within 
minutes. Careless exposure of copper and other marketable materials may encourage 
surreptitious salvage operations. 


Hardware. Factors 


Hardware factors should be evaluated prior to the data-acquisition stage. Instru- 
mentation should be designed for system accuracy and repeatability, neither of which 
should deteriorate under field conditions. Suitable backup for critical data-acquisition 
methods must be provided. Experiments with complex instrumentation may require 
automatic data-acquisition equipment. Precautions for on-the-spot maintenance and 
backup must be taken. Power must be reliable. In some cases, such complex projects 
entail considerable expense for the physical appurtenances necessary for adequate data 
acquisition. The inability to anticipate the magnitude of error inherent in a field mea- 
surement technique may invalidate the data. Inherent error of an inappropriate mea- 
surement technique may approach or exceed in magnitude the change in the test param- 
eter. Some of these considerations may seem quite basic, and they are; but it is sur- 
prising how often they may be overlooked. 


Construction Methods 


Construction methods that constitute accepted highway practice must be considered 
in research design. The investigator can expect to encounter resistance if he proposes 
new methods entailing the use of nonstandard equipment or unfamiliar methods. When 
new methods, procedures, and requirements are included in the contract, shoddy work- 
manship is likely to result unless adequate preparation is made. 


DESIGN COMPENSATION FOR UNCONTROLLED VARIABLES 


Control sections, or sites, are often included in a test area to isolate and evaluate 
the source or cause that may have an influence on the response of the material or con- 
dition being investigated. The number of control sections may be limited by economic 
considerations but ideally should be distributed or selected at random throughout the 
test area, dependent on the variability of the effects to be isolated. Although the main 
test parameter is normally held at a single level for all control sites, additional con- 
trol sites with the test parameter at other levels should be considered if there is rea- 
son to believe an interaction exists between it and uncontrolled sources of variation. 

Measurement of the magnitude of a known uncontrolled variable and compensation 
for its effect may sometimes be successful. The compensation may be applied theo- 
retically if there is known to be good correlation between the variable and its effect on 
the experiment. Usually, however, an empirical correlation is established. In most 
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cases, it is advisable to perform pilot studies in preparation for a full-scale field in- 
vestigation. In this manner, certain conditions that cannot be controlled in the field 
may be synthesized in the laboratory to determine if these uncontrolled conditions 
either cause predictable results or significantly affect the purpose of the research. 
Although pilot tests in the laboratory will result in pseudo data values that may or may 
not be the same as those found in a field investigation, the information obtained can be 
quite valuable when the field experiment is designed. 

Statistical averaging through replication of instrumentation or data acquisition within 
test sites, replication of test sites within an area, and replication of readings by 
different observers or equipment should be considered. Replication and randomized 
treatment provide a means to determine the reliability of acquired data and will tend 
to negate certain uncontrolled variables that are dependent on the site, operator, or 
equipment and will allow statistical treatment of them in the analysis. 

Factorial design using analysis of variance principles allows greater efficiency in 
the use of observations and greater precision with less replication than the classical 
experiment. Random sequences of observations relegate the effects of certain un- 
controlled variables to the residual or error term where they are not confused with 
the test parameters and their contribution to experimental variation can be estimated. 
With the advent of greater computer usage, analysis of variance technique is applicable 
to larger, multivariable investigations. 


SUMMARY 


In summary, there are 4 inseparable phases that must be carefully considered in 
any good field experiment. There are experimental design considering statistical im- 
plications, execution of the field experiment, analysis of data, and implementation of 
findings. Of these, the initial planning and design phase is almost invariably the most 
critical. The damage resulting from a poor design is irreparable; no matter how in- 
genious the analysis, little meaningful information is ever salvaged from data derived 
from poorly planned experiments. On the other hand, if the design is sound, then even 
"quick and dirty" methods of analysis can yield a great deal of useful information. It 
might be stated that the need for careful planning and design is even more acute for 
field experiments than for laboratory experiments. The additional cost of doing field 
work and the necessity to work with individuals who are not normally associated with 
research work make it imperative that all phases of the work be carefully planned and 
coordinated. The experimental design procedures that are commonly accepted in lab- 
oratory work must be extended into field operations if the results are to be meaningful. 
In the California Division of Highways we have learned many of these lessons the hard 
way, primarily through experience. However, through training programs and the in- 
flux of engineers over the past several years with formal training in statistics and ex- 
perimental design, it is now routine to fully consider all the essential elements of 
experimental design, including statistical considerations, from the inception of a 
research project to its culmination in a better highway project. 


Sample Survey Design 
A. L. FINKNER, Research Triangle Institute 


eMANY PROBLEMS in highway transportation and safety are not amenable to controlled 
experimentation, and the investigator must resort to observational studies. In other 
words, the phenomenon under scrutiny must be examined and measured as it exists and 
without disturbing it. Observational studies have less investigative power than con- 
trolled experiments; although relationships can be established, it is often quite difficult 
to assign cause and effect. The techniques used most often to study these phenomena 
are commonly referred to as sample surveys. 

Surveys may be classified into 2 broad general types or groups: descriptive and 
analytic. The investigator in the highway transportation field makes use of both types. 

As it implies, the descriptive survey provides a description of the population from 
which the sample is drawn. A well-designed sample survey not only provides estimates 
of the various characteristics of the population under investigation but also should pro- 
vide estimates of their precision as well. The results of descriptive surveys are often 
used as a basis for administrative action. They also may be exploratory in nature and 
used to develop hypotheses that will be tested by techniques with more investigative 
power. 

An example of a descriptive survey is a litter composition study designed by Research 
Triangle Institute under the auspices of the Highway Research Board and Keep America 
Beautiful, Inc. The data were collected through the cooperation of the state highway 
departments in the various states that participated. Here the objective was to estimate 
the composition of highway litter so that the various industries contributing the products 
that ended up as litter could be made aware of the magnitude of the problem. Although 
the distributions by various domains were compared, the study was not designed to 
measure associations among the various variables. 

Analytic surveys attempt to go beyond pure description and to determine relation- 
ships or test hypotheses. Here, for example, we may wish to test the efficiency of 
driver training or safety belts on reducing accidents or serious injuries. Although it 
might be ideal to be able to select a sample of young men and women of a certain age, 
match them with respect to certain variables, assign them to driver training or no 
training at random, and then measure their safety records under the same conditions 
over a period of years, such an ideal is unattainable. Therefore, we must resort to 
other techniques to obtain answers to the questions raised. Cochran (1) discusses the 
problems in planning observational studies and some of the current strategies in over- 
coming them. The problems can be classified into those encountered in setting up com- 
parisons, dealing with disturbing variables, going from measures of association to the 
elucidation of causation, generalizing from the sample to the population, measuring, 
and considering multiple variables. 

For purposes of illustration, consider 2 problems: (a) comparison of the number of 
serious accidents among safety belt users and nonusers, and (b) effect of various com- 
munity campaigns on the incidence of drinking drivers. 

In the first investigation, we must measure at least 2 quantities. We must first have 
an estimate of the number of serious accidents that occur to both safety belt users and 
nonusers. There are certain problems encountered here. The definition of a serious 
accident may be troublesome. In some accidents the determination will be obvious, but 
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border-line cases may be difficult. There also may be a problem of underreporting 

but, in the main, these problems are tractable. The sample is usually defined in time 
or space or both; i.e., certain areas, such as counties or states, may be selected and 
examined at random times. 

Before any meaningful comparisons are possible, however, an attempt must be made 
to match, at least on major variables. One obvious major variable is the number of 
drivers in the 2 categories and the number of miles driven by each, or the number of 
driver-miles. This is sometimes referred to as exposure and must be estimated from 
an entirely different type of survey. At the time exposure is estimated, other relevant 
variables should be measured. These include descriptions of the driver and vehicle 
and characteristics of the highways and the environment. Of course, these same char- 
acteristics should be enumerated for each serious accident. Even then conclusions may 
be tenuous. For example, if all nonusers are driving old cars that were manufactured 
before seat belts were required, the results could be the result of the age of the car and 
not the failure to use seat belts unless this kind of factor is adequately assessed. Thus, 
in studies of this kind, 2 surveys are required, one to estimate the numerator of a 
variate and one to estimate the denominator; and both are equally important. 

In the second problem, let us consider some proposals being made to test the effec- 
tiveness of various countermeasures to drinking and driving. Howard Pyle, National 
Safety Council president, is quoted as saying that attempts to talk people out of drinking 
and driving have failed. The approach now seems to be to try to teach the people the 
amount of alcohol they can consume and still drive safely. Studies are being designed 
in several major cities in the United States to evaluate the effectiveness of drinking- 
driving countermeasures. Some countermeasures that have been proposed are (a) a 
program of public information and education; (b) a program of strict law enforcement 
in conjunction with special handling of cases by the courts; and (c) programs a and b 
operating simultaneously. It is hoped that the effectiveness of these countermeasures 
can be evaluated and that, if any show promise, they can be adopted on a larger scale. 

First assume that we have only one countermeasure and one city. The so-called 
before-and-after type of study is often employed in situations of this kind. At a mini- 
mum, the incidence of drinking drivers must be estimated and the principal disturbing 
variables such as exposure by various characteristics of the driver, vehicle, highway, 
and environment must also be measured. At an appropriate time after the introduction 
of the countermeasure, the same variables are measured again. This enables us to 
examine whether changes in the drinking driver incidence have occurred over and above 
those expected from changes in the disturbing variables. The estimate of the effective- 
ness of the particular countermeasure is subject to the following 2 types of bias: (a) 
people's behavior immediately prior to the program might be affected by the knowledge 
that the program is about to be initiated; and (b) some disturbing variables that affect 
time changes may be unknown. The first type can be controlled in this case by refusing 
to allow any publicity prior to the introduction of the countermeasure. The second type 
is more serious and may be impossible to control. 

Now, if it is possible to conduct the same program in 2 cities that are somewhat 
evenly matched with respect to disturbing variables, it is possible to select one city at 
random to be the programmed city and the other to be the control. Even if they cannot 
be matched on the disturbing variables, it is usually possible to adjust for such differ- 
ences. Then the differences between the before-and-after measurements in the 2 cities 
are compared. Although we still have the problem of unknown disturbing variables af- 
fecting the 2 cities differently, the probability of success in measuring the effectiveness 
of the countermeasures is greatly enhanced by the addition of the second city. 

When we have only 1 city and 3 countermeasures, like those mentioned earlier, our 
difficulties are compounded. Not only do we have the same problem of unknown disturb- 
ing variables, but we have an unknown contribution from a carry-over effect from the 
previous countermeasure when a new one is adopted. Again, these effects can be best 
measured by adding cities that are as alike as possible and by applying the principle of 
cross-over designs from the field of experimental design. The order of the "treatments" 
or countermeasures must be designed so as to permit estimates of the carry-over 
effects and then the various orders assigned at random to the cities. 
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The estimates of the incidence of drinking drivers also deserves some attention. 
Roadblocks can be set up at random points and at random times. However, the driver 
can refuse to be examined, introducing the possibility of a bias of nonresponse. So long 
as the nonresponse rates are approximately the same in each period of countermeasure 
activity, this bias probably can be ignored. 

In summary, analytic surveys or observational studies should be designed as care- 
fully as possible so as to minimize the bias from potential sources and at the same 
time to allow as much precision as possible. As near as possible, the population 
sampled should be the population about which inferences are to be drawn. If these in- 
ferences are to have validity, probability samples should be used wherever possible. 
Disturbing variables should be identified, and attempts to match or adjust for the most 
important of these should be made. In making the transition from measures of associ- 
ation to causation, the investigator will usually have evidence from a heterogeneous 
collection of results of varying quality. He must weight these results, giving low weight, 
of course, to poor quality information. Finally, he should state his judgments about 
conclusions clearly, attempting to be as objective as possible. 
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Part 2 
Foreword 


The paper by Hong, Schwar, and Talbert is appropriately related to 
the preceding papers in that it discusses the organizational framework 
within which research projects are undertaken. The authors discuss 
concepts and methods for highway transportation research and develop- 
ment programs and present structures for both organizing and program- 
ming research and development within government at all levels and 
within academic, industrial, and other types of organizations. 


A New Approach Toward Formulating 
Research and Development Programs 


H. HONG and J. F. SCHWAR (deceased), Ohio State University; and 
L. O. TALBERT, Ohio Department of Highways 


This paper discusses concepts and methods for structuring R&D efforts in 
highway transportation together with techniques for establishing short- and 
long-range dynamic R&D programs for the purpose of guiding future R&D 
efforts in a systematic and well-coordinated manner. A new R&D organi- 
zation structure is proposed based on the concept of 'mission-oriented 
systems approach." A hierarchical structure of 3 levels is developed in 
terms of strategic, tactical, and action models for the purpose of program- 
ming R&D. These structures are used to develop concepts and methods 
for developing dynamic R&D programs based on R&D needs and resources. 
These program techniques are helpful in providing systemized procedures 
that will ultimately help guide, coordinate, and promote positively defined 
goals. 


®THE MAJOR GOVERNMENTAL AGENCIES, universities, industries, and research- 
oriented organizations with an active interest in highway transportation research are 
confronted with a complex and long-standing challenge, namely, to solve the many- 
faceted problems of highway transportation. What logical research and development 
programs should be formulated to solve these problems? What logical allocation of 
limited resources should be made to support such R&D programs? 

A review of past and current research activities of a number of research organiza- 
tions indicates that (a) each organization tends to have its own method of formulating its 
R&D program, which is generally dependent on the nature of the organization, its poli- 
cies, the internally and externally imposed constraints, and the size and nature of avail- 
able and future resources; and (b) there has been a great deal of fragmentation, duplica- 
tion, and lack of correlation and direction in the overall R&D program and some waste 
of research effort. 

One of the consequences of this unstructured approach to R&D in the past has been 
an absence of tangible benefits commensurate to investment. It is evident that there 
must be better organized and systemized procedures and structures through which the 
entire effort of R&D in highway transportation can be guided, coordinated, and promoted 
toward positively defined goals. This paper is confined to the development of concepts 
and methods of R&D program formulation. These concepts and methods are especially 
important from the viewpoint of subsequent detailed development of R&D programs. 

The problem of R&D program formulation is conveniently examined within the frame- 
work of the following objectives: 


1. To develop concepts and methods for structuring R&D efforts toward improvement 
of highway transportation; and 

2. To develop concepts and frameworks for establishing short- and long-range dy- 
namic R&D programs, guiding future R&D efforts in a systematic and well-coordinated 
manner, and maximizing the benefits resulting from R&D. 


While these objectives can be spelled out only in the broadest terms, the criteria that 
must be met by the resulting R&D programscanbe identified more specifically as follows: 
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1, Provision for both theoretical (new concepts) and applied research (new methods 
and development); 

2. Provision for the translation of theoretical research results into practical use; 

3. Use of systems concepts to permit a better understanding of the relationships 
among many research areas; 

4, Capability of development and expansion within the framework of overall pro- 
grams already established by governmental and industrial agencies; 

5. Realism in terms of availability of physical, fiscal, and manpower resources; 

6. Realism in terms of priority and scheduling; and 

7. Compliance with the research institution's policies. 


The overall significance of a framework within which R&D programs can be developed 
lies in the end product, namely, in a sound base and structure for the orderly expansion 
and development of both existing and future research and development programs. This 
is of particular importance when viewed against the complexity and magnitude inherent 
in future research efforts required to meet the challenges of tomorrow's transportation 
requirements. 

This method of procedure includes 2 phases: (a) development of concepts and 
methods of formulating R&D structures; and (b) development of concepts and methods 
of formulating dynamic R&D programs, These phases, while described separately, 
are not mutually exclusive. On the contrary, they are very much interrelated. 


DEVELOPMENT OF CONCEPTS AND METHODS OF 
FORMULATING R&D STRUCTURES 


A prerequisite to developing R&D programs in highway transportation is to establish 
a number of R&D structures as a guide to the orderly development of such programs. 
There are basically 2 types of R&D structures that are useful toward this end: (a) the 
organization structure and (b) the program structure. 


Organization Structure 


The organization structure is mainly concerned with the arrangement of various, 
often diverse, R&D functions and activities into an integrated, coherent system with 
each part working toward the common organizational goal (in this case, improvement 
of the services provided by the highway transportation system) and each component 
represented in terms of its ultimate performance within some large system. This type 
of systems organization or classification is essential for the development of the R&D 
program structure and, indeed, for successful R&D performance. 

The underlying concept of the evolvement of a highway transportation systems re- 
search structure (or a dominant requirement of this R&D organization structure) is a 
mission-oriented systems approach as shown in Figures 1 and 2. This new approach 
is quite different from the traditional subject-oriented R&D classification (as in the 
Highway Research Board subject organization) in that the various elements that make 
up the research effort are structured to accomplish a positive mission. A structure of 
this nature involves identifying all of the necessary research elements, activities, and 
their relationships; structuring these activities into a proper hierarchical framework; 
and sequencing these activities in such a manner as to be useful toward accomplishing 
a positive mission. Some other characteristics of a mission-oriented structure include 
clear purpose, clear logic, interrelationship, coordination, logical grouping, coherency, 
flexibility, and convergency toward a mission or an objective. 

Based on this concept, 5 system elements of highway transportation systems re- 
search were developed (Figs. 1 and 2). These are administration and management (or 
systems management), systems planning, systems operations, materials, and structures. 
(The system elements of materials and structures could have been combined to form 
"systems physical facilities." The reason for not combining them is that the expertise 
necessary to develop an R&D program in these system elements is quite unique and 
often mutually exclusive; i.e., a materials specialist may not be a specialist in pave- 
ment structure and vice versa.) These 5 system elements not only constitute the essential 
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Figure 1. Structure for highway transportation research. 
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core but also cover the entire spectrum of highway transportation R&D as seen today 
and envisioned for the foreseeable future. These system elements were selected not 
for the purpose of isolating any of the R&D components but for the purpose of forming 
viable systems components that demonstrate the distinct characteristics of highway 
transportation systems research and the need for an integrated synergistic approach 
to R&D program formulation and R&D performance. 

Each system element consists of a number of missions that characterize its vital 
role in highway transportation systems R&D structure. There are a total of 20 mis- 
sions under the 5 system elements (Fig. 2). (Actually, each mission should read, for 
example, "improvement ofthe performance of flexible pavement," "improvement of 
traffic factors," and so on. However, for brevity, only the key words are given.) A 
mission provides a positive research goal to be accomplished. It is an essential hier- 
archical unit that serves as a basis for establishing R&D orientation goals and systems 
integration. A mission also represents expertise in the existing R&D discipline struc- 
ture that could participate in the R&D efforts of its own specialty and capability. 

Each mission, then, is comprised of a number of study areas or problem areas, For 
example, a study area within the traffic factors mission might beimprovements in street 
network operation. This level of the R&D systems hierarchy may be considered as the 
working unit that specifies the positive objectives to be accomplished. An R&D pro- 
gram is formulated to accomplish these specific objectives. These problem areas may 
be of an applied or a theoretical nature. They may be short- or long-range R&D ac- 
tivities. They may be given priorities based on some logical priority criteria. Incor- 
poration of the problem areas into the program structure will be described later. 

Each study (or problem) area is then broken down tothe next lower hierarchy, namely, 
a number of tasks and projects, and structured in terms of phases and flows of R&D 
activities that are required to accomplish the stated objectives. 

In the formulaton of R&D programs on the 
level of tasks and projects under a givenprob- 
lem area, the project or task selection is not 
STRATEGICAL MODEL limited to that particular mission area under 
(POLIGY—DEGISION - MAKING ) which the problem area is considered and for 
which the R&D program is being developed. 

On the contrary, tasks and project selections 
for one particular objective or one problem 
cut across the mission boundaries and thus 
attain the characteristics of a systems ap- 
proach, All of the relevant system components 
Pe sas that define the problem and the major interac~ 
tions are integrated into the problem area R&D 
formulation. 

In summary, an organization strugture pro- 
vides a primary framework and basis for a 
systems approach in both R&D program for- 
mulation and R&D performance. 


ACTION MODEL 
(EXECUTION - PERFORMANCE) Program Structure 


In addition to the R&D organization struc- 
ture described, there is a need for an R&D pro- 
gram structure to describe and to program 
various R&D activities that will achieve the 
stated goals and objectives. An R&D program 
OPERATION MODEL structure, in essence, provides 3 levels of 
SEAYS TSS REY-OFERBIONY hierarchy with respect to the orientation, 
thrust, activities, and structures, The struc- 
ture that describes this hierarchy consists of 
Figure 3. Hierarchy of R&D planning and strategic, tactical, and action models (Fig. 3). 

programming. A fourth model (operation model) is not a 
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program model but a day-to-day R&D operation model and is not considered at this 
time. 

The basic concept underlying this program hierarchy is that, if the basic strategy of 
R&D, either on a national or local level, for a given mission and for highway transporta- 
tion research as a whole is established properly, then there is a strong probability that 
present and future R&D efforts programmed in terms of the tactical and action models 
(which are developed within the framework of the R&D strategy) have at least a fair 
chance to be sound and effective and also may be expected to maximize the return on 
R&D investment. If the basic strategy is not properly formulated, however, the tactical 
and action models would not only be invalid but would also yield the least amount of 
return, 

Thus, it is paramount to develop a series of sound R&D strategies to guide the R&D 
activities on the national level to promising directions and to successful results. The 
task of developing these strategies rests on the administrators of federal and state 
governments and research institutions. These strategies, once established, should be 
reviewed periodically in the light of new technological development and socioeconomic 
constraints and demands, For this reason, dynamic R&D programming is needed to 
revitalize the R&D programs at all 3 levels. The dynamic programming is described 
later. 

The strategic model is developed as follows: 


1. Based on the past and present trends and patterns of R&D in a particular area, 
the state of the art should indicate the status of R&D activities in a given mission or 
missions. Some of the parameters are R&D gaps, fragmentation, duplication, lack of 
positive direction, and general trend. 

2. Based on this information and socioeconomic requirements and constraints, a 
general strategy of future R&D in each mission area can be developed. Future R&D 
strategy may indicate general thrust, orientation, problem areas, goals, priority, con- 
sequences, methodology, and interrelationships. 

3. The general framework of R&D programs should be an optimal framework for 
carrying out future R&D programs indicated in 1 and 2, 

4, This model will be useful to administrators, sponsors, and researchers in that it 
provides an overall picture and strategy of a given mission area and a basis for the 
decision-making process. Thus, basic R&D strategy for a given mission area may be 
established to serve as a guide for the development of tactical and action models, This 
model has a general applicability and utility regardless of geographic location or type 
of organization, 


The second level of hierarchy is the tactical model, which is essentially an R&D 
planning and programming structure, This model is developed within the R&D strategy 
established in the strategic model and has the following characteristics: (a) establishes 
and defines positive objectives, study areas, and problem areas; (b) provides well- 
structured R&D programs for accomplishing objectives and goals established; (c) in- 
dicates various R&D flows, interrelationships, phases, and steps to accomplish the 
stated objectives; (d) serves as an R&D planning and programming tool that is con- 
cerned with the intermediate level of the decision-making processes; (e) is useful for 
R&D management control and for researchers; and (f) has a general applicability and 
utility regardless of geographic location or type of organization. 

The third level of hierarchy is the action model. This defines the tasks indicated 
in the tactical model in relatively finite terms so that various R&D activities may be 
clearly programmed and spelled out and actual R&D may be performed based on this 
program model. This model (a) describes major tasks and projects that are neces- 
sary to place the research activities in clear focus; (b) indicates levels and scope of 
R&D activities of various tasks and projects and interrelationships among them; (c) 
serves as atest model for the hypotheses and directions established in both strategic 
and tactical models; (d) describes the R&D resource requirements necessary to reach 
certain objectives; (e) is useful for R&D management control and researchers; (f) may 
be developed or tailor-made as an R&D program for a particular institution depending 
on its policy, constraints, and resources; and (g) is the basis for actual performance 
(operation model) of R&D. 
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In summary, for any given mission area or a combination of mission areas, the first 
task is to establish the basic strategy with respect to the direction of R&D and to de- 
termine how the stated goals may be achieved. Then, within the framework of this 
R&D strategy, the tactical and action models will be developed to achieve the stated 
goals and objectives in terms of positive R&D programs and resource requirements. 
The strategic and tactical models have general utility independent of geographic loca- 
tion or type of organization, whereas the action model is a tailor-made program for a 
particular organization. 


DEVELOPMENT OF CONCEPTS AND METHODS OF FORMULATING 
DYNAMIC R&D PROGRAMS 


This phase entails the development and evaluation of possible alternative R&D pro- 
grams with the eventual emergence of a final, dynamic program set. 

Basically, there are 2 approaches in formulating an R&D program. One approach 
commonly used is based on the R&D needs. The other, which has not been utilized a 
great deal, is based on available or potential resources. Either approach may be used, 
Preferably the 2 approaches should be combined depending on a number of variables 
and constraints. This 2-way approach concept is shown in Figure 4, 
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Figure 4. Development of model. 
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R&D Needs Approach 


The first step in this approach is to determine the prevailing R&D needs. These 
needs are then categorized into 3 levels of R&D programs, namely, strategic, tactical, 
and action programs (or models), as described in the previous section, 

Several studies related to research needs have been completed in recent years (1, 
2,3, 4). The state-of-the-art literature will be helpful in providing a good overview 
of R&D activities of the present and immediate past. This overview is essential in 
determining the general trend and pattern, gap fragmentation, duplication, and direction 
of R&D. It serves as a basis for formulating the strategic model, whose requirements 
and characteristics were described earlier. R&D needs studies are utilized for deter- 
mining the future R&D needs. By studying programs, reports, and other socioeconomic 
requirements and constraints, future direction, thrust, goals, and priority of R&D are 
established as an integral part of the R&D strategy in each system element and mission 
area. Information from studies is supplemented by further information secured by 
interview and discussion with individuals involved in various research activities in 
highway transportation. Development of R&D strategy is mainly an administrative 
function. 

Based on the R&D goals and priorities established in the strategic model (or pro- 
gram), R&D needs are then formulated into tactical and action programs, The tactical 
program, whose requirements and characteristics were explained earlier, involves de- 
veloping well-structured R&D programs for a specific problem or study area. It in- 
dicates various R&D flows, interrelationships, phases, and steps to accomplish the 
stated objectives. General resource requirements (physical, manpower, and fiscal) 
may be established in this program. Development of the tactical program is an 
administrative-staff function. 

Both strategic and tactical programs are not designed for special usage by any one 
organization. To the contrary, they have a general applicability and utility to most of 
the organizations across the country interested in highway transportation R&D, 

The action program is then developed for the specific set of objectives within a 
problem area (or study area) as defined in the tactical program, This program in- 
volves judicious and logical selection and structuring of tasks and projects required 
for accomplishing the stated objectives. Various resource requirements (physical, 
manpower, and fiscal) necessary for achieving the stated objectives are also indicated 
in detail. Based on this action program, R&D can proceed. The action program is 
tailor-made to suit the particular policy, needs, capability, constraints, and resources 
available to a given organization. Development of the action program is mainly a staff 
function. 

In summary, the R&D needs are first synthesized to form the basic R&D strategy 
along with other requirements for each system element or mission, Then, tactical 
and action programs are formulated of R&D strategy to solve specific problems. 


Resources Approach 


The conventional approach to R&D program development begins with R&D needs and 
formulates the program on the basis of need priorities within the limits of prevailing 
constraints. This approach assumes that all types of resources are equally available 
or that the necessary resources to accomplish certain missions can be planned and 
obtained. This is often not the case. In some areas of research and development, 
there exist unique resources (such as impact sleds, high-speed tracks, and specially 
trained research personnel) that possess very special R&D capabilities and capacities. 
If and when such unique resources exist, it is logical that a special effort should be 
made to formulate the R&D program to make the most efficient use possible of them 
within the confines of the national R&D thrust. This is the underlying principle of the 
resources approach to R&D program formulation, 

Because this approach to program development is built around the special capabili- 
ties of specific resources, it is difficult to explain the procedure in general terms. 
For this reason, the procedure has been illustrated for a specific group of physical 
resources, namely, research laboratories. A similar procedure can be followed for 
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each of the other types of physical resources (such as field installations or tracks) or 


Manpower resources, 


The resulting R&D program developed from such an approach 


will be fully feasible from a resource requirement standpoint and will provide full and 


efficient resource utilization. 


An evaluation of the contribution of laboratories to the total capability of a research 
facility requires a knowledge of the different types of laboratories available and an 
understanding as to how these laboratories fit into the operation plan of such a facility. 
Figure 5 shows an overview of the interactions that take place between physical re- 
sources in the support of a set of research projects. 
interactions are dependent on both the specific character of the research being pursued 
and the type of resources that are available. 

Two general classes of laboratories can be readily identified, namely, primary 


laboratories and secondary laboratories, 


The frequency and extent of these 


Primary laboratories are those directly 


involved in the performance of basic and applied research relating to highway trans- 
portation. Secondary laboratories are essentially support facilities involved along 
with the workshops in providing services to the primary laboratories and the field 
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Figure 5. Study overview. 
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facilities. In general, these laboratories exhibit little or no trade-off with any other 
physical facilities. The primary laboratories by virtue of the direct highway orienta- 
tion have the greatest effect on the overall capabilities of a research facility. For 
this reason they are critical in this approach to R&D program formulation. 

Some of the different types of laboratories currently in operation in industry, at 
universities, and at research institutions in the primary class are highway materials; 
highway structures; optics and illumination; noise and vibration; corrosion and fatigue; 
environmental control; hydraulics; signs, markings, and control devices;.vehicle guid- 
ance; vehicle dynamics and components; traffic dynamics and surveillance; system de- 
sign and system operation; human behavior; medical analysis; impact; fuels and pro- 
pulsion; accident reconstruction and analysis; simulation and simulator development; 
and pollution analysis and control. 

Laboratories in the secondary class include communications, data processing and 
storage, photography, photogrammetry and photo interpretation, instrumentation, 
chemical, radiology, mechanical, electronics, electrical, and strength of materials. 

Each physical resource has a specific capability for meeting different groups of 
research and development needs. The problem is to match the relevant needs with 
the appropriate resource. Once this match has been made it will be possible to for- 
mulate a set of R&D programs that will make efficient use of all available and poten- 
tially available resources. 

Many factors combine to determine the research potential of a primary laboratory. 
Among these are the following: type and amount of specialized equipment available, 
quantity and quality of available manpower, efficiency of laboratory operation, flexi- 
bility of equipment allowing multipurpose usage; and capacity measure (volume of 
R&D that can be handled in laboratory). 

Preliminary study has shown that the first 2 factors listed have the greatest effect 
on laboratory research potential. Hence, it becomes necessary to determine the major 
equipment commonly found in each of the primary laboratories. This will allow pre- 
vailing research needs (on a study area, project, or task level) to be related to the in- 
dividual research capabilities of the laboratories with a greater degree of certainty. 
Once this has been done research programs based on research capabilities can be de- 
veloped to make efficient use of all laboratory facilities as has been previously sug- 
gested, At the same time guidelines may be set for the acquisition of needed facilities 
in areas where no suitable facilities exist. 

The selection of the proper physical resources for inclusion in a research facility 
is a difficult problem. This problem is complicated in the case of research labora- 
tories because the research potential of such laboratories is a function of the avail- 
ability of both equipment and manpower. The level of funding available further con- 
strains the laboratory choice. 

Figure 6 shows in flow-chart form a general model developed for use in making the 
laboratory selection. This model combines decision-making processes in 3 different 
levels of hierarchy: the strategic level, the tactical level, and the action level. The 
initial input to the model is a set of needs that have been selected from the universe 
of needs on the basis of similar resource requirements. Once a needs-area has been 
selected for study it must undergo a policy review and a technical review to establish 
its validity. The purpose of the policy review is to determine whether the selected 
needs area (a) falls within the scope of interest of the parent research agency, (b) is 
in line with long-range policies and goals, and (c) offers sufficient potential payoff for 
inclusion in the R&D program, All needs areas that satisfy these 3 criteria are sub- 
jected to technical review. Needs that do not meet the criteria are dropped from con- 
sideration. 

The purpose of the technical review is (a) to determine whether the selected needs 
area is still valid in view of present state of the art; (b) to determine whether the re- 
search in a selected area is within the technical capabilities of the parent research 
agency; and (c) to identify special problems involved in selected research that will re- 
quire special attention and effort from the research staff. After the technical review, 
all research areas judged satisfactory can be included inthe R&D program. Those 
judged unsatisfactory are dropped from consideration. 
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Figure 6. General model for resource selection. 


The R&D program derived in this manner represents a compilation of study areas 
deemed worthwhile from both policy and technical standpoints. It is important to note 
that simply because a study area is on the program does not mean that it will be im- 
plemented in the short run. In order to be implemented, a study area must be moved 
from the program into the current work plan and defined more specifically in terms of 
a set of research projects. The impetus for such a move may come from any one of 
a number of causative factors, which have been termed "initial movers" because they 
supply the initial push. Prominent among these initial movers are the following: high 
national priority, promise of high reward, special interest by resident staff, available 
facilities, work shortages, and special policy decisions. 

The exact nature of the initial mover of a particular project is important because it 
determines the extent to which physical facilities will be allocated to that project. For 
example, a project with a high national priority that promises high rewards may justify 
the allocation of large numbers of physical resources to bring about its swift comple- 
tion, whereas a project generated because of special interests of individual researchers 
might be allocated lesser resources. 

Once the initial impetus has been applied, a project must pass a second policy re- 
view, and sufficient funding must be found before it can be included in the work plan. 
The policy review at this level is primarily concerned with establishing priorities 
relating the project in question to other projects in the work plan. The priority as- 
signed will determine the level of capital improvement funds that can be released to 
acquire needed physical resources among which are primary laboratories. The type 
of laboratory chosen depends on the detailed requirements of the selected project and 
the nature of other facilities already existing or planned. The concept of trade-off 
must also be considered. The size of the laboratory depends on the initial mover 
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concept, the quantity of manpower available, the amount of capital improvement funds 
available, and the size of other existing or planned facilities. 

Once the selection has been made, pertinent information on research capability is 
fed back into the program development process, and the program is modified accord- 
ingly. In this manner the resulting R&D program is both dynamic and accomplishable. 

Insummary, anR&D program may be developed from either R&D needs or resources 
or from both. The initial concepts and methods are developed. They are to be refined 
during the course of further study. 


Dynamic Programming of R&D 


Once the R&D strategy and the tactical and action programs are formulated, R&D 
can be carried out and useful results obtained. However, any set of R&D programs 
so established may become partially or totally obsolete in time because of new tech- 
nological development and varying socioeconomic requirements and constraints. For 
these reasons any R&D program set must go through a series of revitalization pro- 
cesses, more commonly known as a dynamic programming process. There are basi- 
cally 3 types of R&D dynamic programming: internal, external, and internal-external. 

The internal dynamic programming is applicable to an R&D area where R&D ac- 
tivities are relatively well defined in terms of objectives, activity elements, sequences, 
and feedback so that the R&D program may be formulated to possess inherent dynamic 
characteristics. Thus, under a given R&D strategy, the tactical (or tactical-action) 
program is formulated in terms of a series of R&D objectives, flows, and element in- 
terrelationships sequencing with a proper logic and feedback to have internal dynamic 
programming built in. Administrative control and monitoring, however, are still 
necessary. 

The external dynamic programming, on the other hand, is more applicable to an 
R&D area where the R&D activities may not be readily defined or programmed with 
any reasonable certainty. Therefore, there must be some external mechanism that 
will make a given R&D program dynamic. The general concept and method of an ex- 
ternal dynamic programming of R&D in highway transportation are shown in Figure 7. 
The basic concept is a periodic feedback and review of 3 programming models. It is 
recalled that the first task is to establish a sound R&D strategy. Then, within the 
framework of this strategy, tactical and action programs are developed. Thus, it is 
necessary to make a systematic reappraisal and review of the basic R&D strategy in 
the light of various dynamic requirements and constraints. For the purpose of this 
study, a 6-year review by administrative action is proposed. Detailed functions and 
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Figure 7. Dynamic programming of R&D in highway transportation. 
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activities are described in the diagrams, Then, within the framework of a revised 
R&D strategy, the tactical and action programs are reviewed and revised. The tactical 
program is revised every 3 years and the action program every year. The results of 
evaluation, reappraisal, and revision of these 2 programs are then fed into the strategic 
program to verify whether the strategy requirements and constraints are satisfied. 
Thus, an R&D program may be reoriented and revitalized to solve the stated problems. 

The internal-external dynamic programming is a hybrid of the 2 previous methods. 
This method is useful for formulating a short-range R&D program by utilizing the 
internal dynamic programming method and for formulating a long-range R&D pro- 
gram by utilizing the external programming method when a large R&D program 
consists of both short- and long-range activities. 


SUMMARY 


In essence, this paper attempts to provide some guidance and direction to R&D pro- 
gram formulation. Emphasis is placed on the concepts and methods of formulating and 
developing R&D programs, The new concepts and methods are based on a mission- 
oriented systems approach. Based on this concept, various R&D program structure 
methods are developed. These concepts and methods provide a basis for better or- 
ganized and systemized procedures and structures through which the entire effort of 
R&D in highway transportation may be guided, coordinated, and promoted toward posi- 
tively defined goals. 
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HE NATIONAL ACADEMY OF SCIENCES is a piivate, honorary organization: of 

more than 700: scientists and: engineers elected on the basis’ of outstanding 

contributions to knowledge, Established by a Congressional Act of Tncorpora- 
tion signed by Abraham Lincoln on March 3,/1863, and supported by private and 
public funds, the) Academy works to further-science’ and its use for the: general 
welfare by bringing together the most) qualified individuals to deal with scientific 
and technological problems. of broad significance: 


Under the terms of its Congressional charter, the Academy is also called upon 
to actjas. an- official—yet independent—-adviser to the Federal Government in any 
matter of science and technology. This provision accounts for the close ties’ that 
have always existed between the Academy and’the Government; although ‘the 
Academy is not a governmental agency and its activities are not limited to those 
on behalf of the Government, 


The NATIONAL ACADEMY OF EEINGINBERING was’ established 6n December 5, 1964. 
On'‘that date the Cotineil of the National Academy of Sciences, tinder the authority 
of its Act of Incorporation, adopted Articles of Organization bringing the Na- 
‘tional Academy of Engineering into being, independent and autonomous in its 
organization and the election of its members, and closely coordinated! with the 
National Academy of Sciences in its advisory activities. The two Academies join 
inthe furtherance of science and engineering and: share the responsibility), of 
advising the Federal Government, upon request, on any subject of science ‘or 
technology, 


The NATIONAL RESEARCH COUNCIL was. organized as an agency of the National 
Academy of Sciences in 1916, at the tequest of President Wilson, to enable the 
broad: community of US, selentists and engineers to associate their efforts with 
the limited membership of ‘the Academy in service to science and the nation, Its 
members, who receive their appointments from the President of the National 
Academy of Sciences) are drawn from academic, industrial, and government 
organizations throughout the country, Phe National Research Cotincil serves both’ 
Academies inthe discharge of their responsibilities. 


Supported by? private and public contributions, grants, and contracts, and 
voluntary contributions of time and effort by several thousand of the nation’s 
leading’ scientists and engineérs, the Academies and their Research Couneil ‘thus 
work to serve the national interest, ta foster the sound development: of science 
and engineering, and to promote their effective application for the benefit. of 
society. 


The DIVISION OF WNGINEERING is one of the eight major Divisions into which 
the’ National Research Council is ‘organized’ for the conduct of: its avork, Its 
membership inchides representatives ofthe nation’s leading ‘technical gocieties as 
well as’a number of members-at-large, Its Chairman is appointed by the Council 
of the Academy Of ‘Sciences upon nomination! by the Council of the Academy of 
Engineering. 


The HIGHWAY RESEARCH BoarD, an agency of, the Division of Engineering; 
was established November 11, 1920, asa cooperative organization of the highway 
technologists of Amerita operating under the auspices: of the National Research 
Counéil-and swith. the siipport of the several highway departments, the Bureau of 
Public Roads, and many other- organizations interested inthe development of 
transportation. The purpose of the Board is to advance knowledge concerning 
the nature and performance. of transportation systems, through the stimulation 
of fesearch and dissemination of information derived therefrom. 
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